Nature Methods

Quantitative studies of plant growth and environmental responses increasingly rely on time-series imaging, yet automated segmentation remains challenging due to continuous growth, large non-rigid morphological change, and frequent self-occlusion. Traditional image-processing pipelines and taskspecific deep learning models often require extensive annotated datasets and retraining, limiting portability across species, developmental stages, and imaging conditions. Here we present SAP (Segment Any Plant), a plant-focused framework that leverages the pretrained Segment Anything Model 2 (SAM2) to enable few-shot, training-free segmentation of plant timeseries imagery. SAP integrates interactive prompting, automated temporal mask propagation, and centerline extraction within a web-based interface, allowing users to move from raw images to quantitative descriptors of organ shape and dynamics without programming expertise. Across multiple systems, including Arabidopsis thaliana rosette development, root growth, sunflower gravitropism, and confocal root microscopy, SAP achieves high segmentation accuracy (mean IoU 0.890.93) and sub-pixel centerline precision from single-frame prompting. By reducing the need for task-specific retraining, SAP provides a transferable framework for reproducible time-series phenotyping across diverse experimental contexts.

Capturing the three-dimensional (3D) organization of cells is essential for deciphering complex biological processes, yet comprehensive 3D spatial omics is severely hindered by the destructive nature of physical sectioning and the depth limitations of intact tissue imaging. Current computational methods rely on 2.5D stacking of discrete slices, which inherently disrupts tissue topology and fails to resolve continuous depth-dependent molecular gradients. To bridge this gap, we introduce DO_SCPLOWEEPC_SCPLOWSO_SCPLOWPATIALC_SCPLOW, an Optimal Transport flow matching framework that models tissue evolution as a continuous dynamic vector field. By solving the underlying probability flow ODEs, DO_SCPLOWEEPC_SCPLOWSO_SCPLOWPATIALC_SCPLOW enables the direct extraction of uninterrupted, infinitely resolvable tissue states at arbitrary spatial depths. Using Deep STAR/RIBOmap 3D technologies, we demonstrate that DO_SCPLOWEEPC_SCPLOWSO_SCPLOWPATIALC_SCPLOW achieves improved 3D reconstruction fidelity relative to 2.5D approaches, yielding structures that more closely recapitulate native tissue microenvironments in real-world datasets. Across diverse spatial omics modalities, including spatial proteomics using imaging mass cytometry in human breast cancer and spatial transcriptomics using openST in head and neck squamous cell carcinoma metastatic lymph nodes, DO_SCPLOWEEPC_SCPLOWSO_SCPLOWPATIALC_SCPLOW produces biologically interpretable and high-fidelity reconstructions across datasets. We evaluated the scalability and robustness of DO_SCPLOWEEPC_SCPLOWSO_SCPLOWPATIALC_SCPLOW on a large-scale mouse brain dataset, reconstructing a continuous 3D cellular atlas comprising 39 million cells within 41.6 hours. Systematic downstream characterization validated its ability to recapitulate consistent spatial architectures, cell-type distributions, transcriptomic patterns, and microenvironmental structures across brain regions. Collectively, these results demonstrate DO_SCPLOWEEPC_SCPLOWSO_SCPLOWPATIALC_SCPLOW as a generalizable and efficient solution for true 3D spatial reconstruction across scales and modalities. O_FIG O_LINKSMALLFIG WIDTH=200 HEIGHT=117 SRC="FIGDIR/small/721395v2_ufig1.gif" ALT="Figure 1"> View larger version (49K): org.highwire.dtl.DTLVardef@1a19624org.highwire.dtl.DTLVardef@188361forg.highwire.dtl.DTLVardef@199321corg.highwire.dtl.DTLVardef@a8f411_HPS_FORMAT_FIGEXP M_FIG C_FIG

Deep learning can extract quantitative measurements from microscopy images that are inaccessible to classical analysis, but developing these models requires machine learning expertise that most imaging scientists do not have. Here we present a framework in which a researcher describes their microscopy problem to a large language model (LLM) agent in under ten minutes of conversation--specifying what they image, what they want to measure, and what success looks like--and the agent autonomously handles the rest: designing physics-based training data, implementing a neural network, training, diagnosing failures, and iterating without human intervention. A researcher can start the agent before leaving the lab; overnight, it tests tens to a hundred model variations, each one an experiment that would otherwise demand active attention. We validate the framework across six microscopy modalities and four problem types. On the BBBC039 nuclear segmentation benchmark, the agent autonomously trains a U-Net with 3-class semantic segmentation and morphological post-processing, achieving pixel-level Dice of 0.97 and object-level F1 of 0.84--within 7% of the published baseline--while diagnosing a data pipeline bug that no amount of hyperparameter tuning could resolve. On single-protein holographic microscopy, the agent reads a published paper, designs a simulator, and develops an optimized model in a single session. On PatchCamelyon histopathology classification, the agent autonomously evolves through four optimization phases--from scratch training through transfer learning and regularization to inference-time ensembling--completing 97 iterations on 262,144 images to reach 89.3% test accuracy and 96.3% AUC, nearly matching the published rotation-equivariant baseline. This framework enables microscopy researchers to use deep learning-based image analysis without machine learning domain knowledge.

Correcting segmentation errors in large-scale 3D nuclei reconstructions requires reasoning about which fragments belong to the same nucleus across densely packed regions. Existing correction methods rely on local pairwise fragment matching, which cannot resolve the global topology of nuclear clusters and fails to recover missing morphology. We propose NO_SCPLOWUC_SCPLOWGO_SCPLOWRAPHC_SCPLOW, a graph-based reasoning framework that operates over atomic 3D primitives obtained by decomposing erroneous masks. NO_SCPLOWUC_SCPLOWGO_SCPLOWRAPHC_SCPLOW encodes primitive geometry via a 3D point-cloud backbone and performs global relational reasoning through graph attention, capturing inter-primitive dependencies across entire clusters rather than isolated pairs. A primitive-proposal contrastive loss aligns local primitive features with nucleuslevel semantics, improving grouping accuracy in dense regions. The resulting proposals are then refined by a shaperefinement network that predicts signed distance fields to restore smooth morphology. To train without manual error annotations, we develop a self-supervised data engine that synthesizes realistic segmentation errors from clean nuclei labels. To benchmark correction at brain scale, we curate NucEMFix, the first brain-wide EM benchmark of nuclei error cases across FAFB and MICrONS (8,000+ annotated error nuclei). NO_SCPLOWUC_SCPLOWGO_SCPLOWRAPHC_SCPLOW attains 87.99% F1 on NucEMFix-F (FAFB) and 86.20% on NucEMFix-M (MICrONS), outperforming both re-segmentation baselines (e.g., +8.6% over nnU-Net) and pairwise correction methods, while reducing curation effort by over 100x relative to manual proofreading. Code and data are available at https://mingzhiwang618.github.io/NucEMFix.

Parallel revolutions in intravital microscopy and spatial biology techniques have respectively enabled large-scale recordings of cellular dynamics in live animals and multi-dimensional molecular profiling at single-cell resolution. However, due to the challenges of aligning data from different modalities at cellular resolution, these two transformational approaches have generally been applied on separate biological samples, stymying the ability to link activity patterns and molecular attributes in the same exact cells. To enable routine, multimodal investigations of cells in vivo dynamics and molecular content, we created TRU-FACT (Total Registration Under Functional Activity, Connectivity, and Transcriptomics), a broadly applicable experimental and computational pipeline for registering large populations of individual cells across intravital imaging and spatial biology datasets. The pipeline combines three key innovations: an optomechanical tissue handling and alignment method to parallelize specimen planes, a graph-theoretic method to register individual cells based on their geometric relationships to neighboring cells, and a statistical framework that provides for each cell an a posteriori probability of correct registration. We validated TRU-FACT with several preparations for imaging neural Ca2+ activity in cortical and deep brain areas in head-fixed and freely behaving mice, RNA-barcode-expressing viruses for labeling neural projections, and low- and high-plex spatial transcriptomic methods. In mice performing a skilled reaching task, TRU-FACT alignments revealed the movement-related signaling patterns of intratelencephalic, extratelencephalic, and striatum-, superior colliculus-, and thalamus-projecting motor cortical neurons. Overall, TRU-FACT constitutes a scalable, multimodal discovery platform that is applicable to diverse tissue-types and spatial biology techniques, thereby enabling multiscale analyses of many complex biological systems.

Transcriptomics has transformed our understanding of the brain, but assigning transcriptomic identities to neurons recorded in vivo remains challenging at scale. Existing platforms can pair transcriptomic identity with two-photon calcium imaging in small populations of approximately 100 neurons, but they require recorded cells to be sparse and therefore cannot be applied to large population recordings. Here, we present coppaFISH 3D, a spatially resolved transcriptomics method, and CASTalign, an in silico alignment framework, which together enable transcriptomic identification of thousands of simultaneously recorded cells. coppaFISH 3D detects hundreds of genes in thick 50m fixed sections while preserving tissue integrity, enabling both 3D registration to in vivo imaging and integration with immunofluorescence labelling. The platform is fully powered by open chemistry and open source software, runs on commodity hardware, and can be performed at very low cost per section. It therefore enables transcriptomic identification of recorded neurons at scale, making it possible to study how transcriptomic identity shapes activity in neural populations.

Inferring continuous morphological transformations from collections of static biological snapshots is an important, yet challenging problem. In the context of cellular biology, prevailing approaches reduce 3D shape collections to static reconstructions or hand-crafted descriptors, which fail to capture smooth, multi-dimensional transitions. We present MorphCurveVAE, a two-stage pipeline for constructing continuous morphological trajectories from sets of static, segmented 3D microscopy images. Stage 1 learns a smooth, compact latent manifold of volumetric morphologies using a multi-branch convolutional variational auto-encoder (VAE) that can encode multiple correlated substructures into disentangled subspaces. Stage 2 extracts a constrained, topologically-aware principal curve through the augmented latent space to produce directional and correlated trajectories of structural dynamics. To demonstrate our framework, we apply MO_SCPLOWORPHC_SCPLOWCO_SCPLOWURVEC_SCPLOWVAE to a large public dataset (Allen Institute WTC-11) of segmented volumetric cell and nucleus images spanning the mitosis cycle. Our results indicate high-quality reconstructions, low projection errors to the fitted principal curve, and biologically and visually plausible continuous animations. These results suggest MorphCurveVAE as a practical tool for modeling biological morphological trajectories, while remaining broadly applicable to other biological imaging domains where time-resolved observations are unavailable.

Advances in tissue clearing and light-sheet microscopy enable cellular resolution whole-brain 3D imaging. However, whole-brain quantification tools do not yet meet demands for efficiency or assess morphometry. Here we present CellPheno, a 3D nuclei instance segmentation framework for high-throughput cellular phenotyping. CellPheno quantifies an entire P4 mouse brain within 15 hours. We showcase whole-brain morphometry, enhanced stitching, and co-localization across multiple cell types in 53 brains.

Spatially resolved omics technologies reveal tissue organization at single-cell resolution but remain limited by the cost of the assays, incomplete spatial coverage, 2D-only imaging, and experimental artifacts. These factors motivate the need for in silico methods that can reconstruct or extend tissue context beyond what current spatial measurements provide. We present MORPHE (MOdeling of stRuctured sPatial High-dimensional Embeddings), an AI framework that learns to synthesize biologically faithful tissue architecture directly from spatial-omics data. MORPHE introduces a graph-informed probabilistic embedding that maps discrete cell identities and their spatial relationships into a continuous RGB-like latent space compatible with diffusion modeling. This representational bridge enables spatial cellular maps to leverage large pre-trained image-generative models while preserving biological interpretability upon decoding. By modeling cells as the fundamental units of generation and learning how their identities and spatial relationships collectively give rise to large-scale tissue structure, MORPHE enables generation and reconstruction of tissue architecture at single-cell resolution. We applied the method across large-scale single-cell proteomic datasets from the intestine and single-cell transcriptomic datasets from the brain, showing computational scalability acrosss millions of cells. We used MORPHE on these datasets to outpaint beyond experimentally restricted fields of view, inpaint missing or experimentally damaged tissue regions, and perform cross-tissue imputation, connecting separated tissue regions into a single contiguous sample in both 2D and 3D. MORPHE represents a new class of tissue generation algorithms that will help solve current limitations and challenges with single-cell spatial-omics datasets.

Many fundamental mammalian brain functions emerge from the coordinated activity of cells distributed across large, brain-wide networks. To understand these processes in healthy and diseased states, ideally one would simultaneously measure and analyze single-cell activity at the brain-wide scale, an enduring challenge for live-measurement approaches that often face an inherent tradeoff between spatial resolution and scale. Here, we present GLOBE (sinGle-cell spatiotemporaL recOrding Brain-widE), a technology for brain-wide single-cell recording of cellular activity in vivo with spatiotemporal resolution, physiological sensitivity, and parallelization-accelerated readout. GLOBE leverages genetically encoded intracellular protein tape recorders and a high-throughput computational platform for integrated image and signal analysis. GLOBE records analog signal amplitudes across a continuous time axis, requires only standard light microscopy for in situ readout, and is compatible with expansion microscopy and RNA readouts. We applied GLOBE to simultaneously record transcriptional activity of the immediate early gene Fos in up to 219,703 neurons simultaneously across a single mouse brain over 5.5 continuous days, with a timestamp precision of 3.1-6.7 hours (median absolute error), a local recording density of 69-90% of neurons per imaging field of view, and a post-mortem imaging readout speed of 2.9 seconds per neuron on average. GLOBE resolves the brain-wide spatiotemporal structure of single-cell activity, revealing that Fos transcriptional dynamics associated with fear learning and memory retrieval are distributed across the brain with region-specific temporal heterogeneity, and that the variance of this structure scales down as the number of sampled cells increases. We envision GLOBE to have broad applications for dissecting and decoding physiological and pathological processes at the brain-wide scale.

Multiplexed fluorescence imaging is limited by spectral overlap and the small number of excitation or emission channels available on standard microscopes, restricting most laboratories to low-plex imaging. Here we introduce physics-informed spectra-free multiplexed imaging (PhySMI), a self-supervised framework for underdetermined spectral unmixing that enables highly multiplexed imaging without dense spectral measurements after training. By embedding the spectral forward-mixing process into a self-consistent architecture, PhySMI recovers physically plausible source decompositions from unlabeled data without paired ground-truth labels while suppressing stochastic acquisition noise. PhySMI resolves five subcellular structures from only three excitation channels, overcoming the conventional channel-number limit while preserving spectral fidelity and minimizing crosstalk (<0.5%). The framework also generalizes across imaging systems, enabling zero-shot deployment on standard fluorescence microscopes. In live cells, PhySMI enables fast five-color imaging of dynamic multi-organelle interactions with improved temporal resolution and reduced photobleaching and phototoxicity relative to conventional spectral imaging. These results establish a general strategy for physics-informed learning in underdetermined imaging inverse problems and represent a step toward a general-purpose framework for highly multiplexed fluorescence imaging on standard microscopy platforms.

The rapid growth in microscopy data volume, dimensionality, and diversity urgently calls for scalable and reproducible analysis frameworks. While efforts on the open OME-Zarr format have helped standardize the storage of large microscopy datasets, solutions for standardized processing are still lacking. Here, we introduce two complementary contributions to address this gap: 1) the Fractal task specification, defining OME-Zarr processing units that can interoperate across computational environments and workflow engines, and 2) the Fractal platform, using this specification to enable scalable and modular OME-Zarr-native analysis workflows. We demonstrate their use across diverse biological research data, including terabyte-scale multiplexed, volumetric, and time-lapse imaging. In a clinical setting, we show that Fractal workflows achieve near-identical quantification of millions of cells across independent deployments, demonstrating the reproducibility required for translational applications. With its growing community of contributors, the Fractal ecosystem provides a foundation for FAIR microscopy image analysis relying on open file formats.

Computational methods have advanced the analysis of animal behavior, yet significant challenges remain in data standardization, analytical reproducibility, and workflow integration. Existing computational solutions often demand extensive programming proficiency or compel users to navigate a highly fragmented ecosystem of disconnected tools for tracking, statistical analysis, and visualization. Here, we present EthoClaw, an open-source, artificial intelligence-driven workflow platform built upon the OpenClaw agentic framework, functioning as a locally deployable AI assistant for behavioral research. EthoClaw provides an integrated computational infrastructure that seamlessly bridges the gap between raw behavioral video acquisitions and publishable scientific results. In this study, we demonstrate the platforms capacity to natively ingest video data via a dual-mode tracking architecture: utilizing ultra-fast image processing for rapid object detection, and leveraging the SuperAnimal methods for precise, markerless postural tracking. To ensure maximal interoperability, EthoClaw automatically converts various tracking data formats into DeepLabCut-compatible formats, enabling high-throughput phenotyping by generating publication-quality visualizations alongside rigorous multidimensional statistical profiling. Furthermore, the platform incorporates a large language model (LLM)-driven reporting module that dynamically synthesizes analytical documents, ensuring methodological transparency. Through an open field test, we validate the practical usability of EthoClaw while accelerating computational throughput by localizing heavy video processing to circumvent cloud bandwidth bottlenecks. Operating via an omnichannel natural language interface that integrates seamlessly with ubiquitous instant messaging software, EthoClaw democratizes advanced computational behavioral analysis, offering a holistic, highly efficient ecosystem that enforces experimental reproducibility and open science principles.

Connectomic reconstruction from large image volumes produces segmentation and synaptic-assignment errors that must be resolved to support downstream analyses. As datasets have grown larger and teams more distributed, proofreading has become a critical operational bottleneck. Workflows for proofreading and error correction have not scaled commensurately with connectomic data production and may not accommodate heterogeneous proofreader expertise and machine-generated candidate edits. New tools are therefore needed to organize, prioritize, and coordinate proofreading at volume scale. Here we present NeuVue, a task-management and prioritization framework that operationalizes proofreading through atomic, auditable tasks for individual and team review, multistage routing across proofreader cohorts, performance and volume-state tracking, and integration with community annotation, visualization, and analysis services. We report the use of NeuVue across two volumetric datasets, supporting scalable proofreading by over forty proofreaders and producing over fifty thousand edits. NeuVue provides a reproducible human-in-the-loop framework for generating, validating, and maintaining large connectomic datasets.

Multi-view pose estimation is essential for quantifying animal behavior in scientific research, yet current methods struggle to achieve accurate tracking with limited labeled data and suffer from poor uncertainty estimates. We address these challenges with a flexible framework that can operate with or without camera calibration, combining novel training and post-processing techniques with an uncertainty-aware pseudo-labeling distillation procedure. Our multi-view model processes all camera views jointly using a pretrained vision transformer backbone, and a simulated occlusion technique encourages the model to learn robust cross-view correspondences without requiring camera parameters. When camera parameters are available, 3D data augmentations and a triangulation-based loss further encourage geometric consistency. We extend the Ensemble Kalman Smoother (EKS) post-processor to the nonlinear case, leveraging camera geometry, and introduce a variance inflation technique that detects cross-view inconsistencies and corrects overconfident predictions. We validate our approach on five datasets spanning three species (fly, mouse, bird), including a multi-animal dataset with two visually distinct individuals; the proposed pipeline consistently outperforms existing methods across datasets. We demonstrate how these improvements translate to downstream scientific analyses using data from the International Brain Laboratory, showing improved unsupervised behavioral clustering and neural decoding of paw kinematics with just 200 labeled frames. To facilitate adoption, we developed a browser-based, cloud-compatible user interface that supports the full life cycle of multi-view pose estimation, from labeling and model training to post-processing with EKS and diagnostic visualizations.

Cryo-electron microscopy (cryo-EM) is the mainstream method for structure determination, yet current automated workflows remain rigid and require expert intervention for failure recovery, heterogeneity analysis, and optimization. We present cryoAgent, an agentic workflow for autonomous cryo-EM image processing with adaptive tool use to address these challenges. cryoAgent improves reconstruction quality across diverse datasets, identifies a previously unreported structural state, and outperforms state-of-the-art automated workflows, advancing scalable and discovery-oriented structural biology.

Multiphoton microscopy is increasingly pushed toward deep, large-scale and fast functional imaging, where acquisition speed has become a bottleneck. Here we present SPARC, a structure-guided, physics-aware reconstruction framework for scan-limited multiphoton imaging. Rather than treating undersampled recovery as generic super-resolution, SPARC formulates it as a constrained reconstruction problem tailored to point-scanning microscopy by incorporating a sample-matched structural reference from the same field of view and the anisotropic acquisition physics of scan-limited imaging. SPARC jointly integrates denoising and deep upsampling, enabling stable recovery from noisy sparse measurements. In simulated calcium imaging with ground truth, SPARC improved spatial reconstruction fidelity and temporal signal recovery relative to existing methods. In vivo, SPARC improved temporal readout in three-photon and mesoscale two-photon calcium imaging, and enabled a more temporally informative 400-Hz voltage readout on a standard resonant-galvo two-photon microscope. These results suggest SPARC expands the functional operating range of existing multiphoton microscopes under scan-limited conditions.

Quantitative fluorescence microscopy is frequently confounded by spatially varying illumination and temporal intensity drift. Although BaSiC is a widely adopted retrospective correction method, it can fail when foreground content is strongly correlated across images--a common regime in time-lapse, tiled and volumetric acquisitions--and its application often requires manual parameter tuning that limits reproducibility and scalability. We introduce BaSiCPy, a foreground-aware implementation of BaSiC that improves illumination profile estimation under correlated foreground structures, provides automatic hyperparameter selection and accelerates large-scale processing through GPU support. BaSiCPy is distributed as an open-source Python package with graphical and programmatic interfaces, facilitating integration into contemporary bioimage analysis workflows.

Protein-protein interactions (PPIs) mediate diverse cellular processes, but PPIs are typically characterized using reconstituted in vitro biochemical and biophysical approaches. Current approaches for PPI detection in living cells are limited in the scope of interactions they can capture and often require prior knowledge of the interacting partners. To close this gap, we developed triplet tumbling microscopy (TTM), which reveals the interactions of a tagged protein of interest in cells in real time. TTM reports protein complex size from rotational diffusion ("tumbling") by leveraging infrared-triggerable emission from triplet states to track tumbling over nanoseconds to hundreds of microseconds. These long-lived triplets overcome the size limitations of existing rotational diffusion-based approaches, enabling TTM to measure species from small protein complexes to organelle-scale beads. In living cells, we apply TTM to detect PPIs, quantify fraction bound, and distinguish protein complexes by size. We measure diverse types of interactions, including rapamycin-induced dimerization, p53 homo-oligomerization, and binding of the E3-ligase E6AP to the human papilloma virus 16 E6 protein. The required hardware is compatible with most fluorescent microscopes, making TTM a versatile way to extract molecular insights from the complex context of living cells. O_FIG O_LINKSMALLFIG WIDTH=200 HEIGHT=109 SRC="FIGDIR/small/723557v1_ufig1.gif" ALT="Figure 1"> View larger version (27K): org.highwire.dtl.DTLVardef@1e70768org.highwire.dtl.DTLVardef@974813org.highwire.dtl.DTLVardef@1fd122borg.highwire.dtl.DTLVardef@1b3da96_HPS_FORMAT_FIGEXP M_FIG C_FIG

The rapid convergence of advanced microscopy and deep learning is transforming cell biology by enabling imaging systems in which optical encoding and computational inference are jointly optimized for volumetric information capture and interpretation. However, broadly accessible three-dimensional imaging at high spatiotemporal resolution remains constrained by volumetric reconstruction throughput, susceptibility to artifacts, and the burden of collecting modality-matched training data. Here, we introduce PAVR, a physics-aware light-field imaging platform that integrates single-shot volumetric acquisition with fast, end-to-end volumetric reconstruction. PAVR is trained entirely using in silico system responses, avoiding reliance on external high-resolution ground-truth modalities and enabling sample-independent reconstruction across diverse biological contexts. Using fixed and live mammalian cells, we demonstrate multicolor volumetric imaging of subcellular organelles, three-dimensional tracking of autofluorescent particles, and high-speed visualization of organelle remodeling and interactions. We further extend PAVR to quantify coupled morphological and functional dynamics in beating human induced pluripotent stem cell-derived cardiomyocytes under pharmacological perturbation. Together, PAVR establishes a scalable hardware-software platform for high-throughput volumetric imaging and quantitative analysis of dynamic cellular systems in both basic and translational settings.